Conversation
upload_file took remote_path verbatim and download_file took an untrusted local_path verbatim, so a caller passing '../../home/aila/.ssh/id_rsa' could read or overwrite files outside the intended transfer directory. A segment-scoped guard now rejects any path with a '..' traversal component (POSIX or Windows separator) or an empty path, on the remote side of uploads and both the remote and local sides of downloads, before any SFTP connection is opened. Normal relative and absolute paths (including hidden-dir targets and 'foo..bar' names) are unaffected.
The LLM client retry loop logged and re-raised provider exception text verbatim, so an Authorization: Bearer token, an api_key= query parameter, or an inline sk-* key echoed in a provider error landed in the worker log and the propagated LLMError message. A new redact_secrets helper in the C6 log_redact boundary (covers bare 'bearer <token>' and inline sk-* keys in addition to the existing command markers) masks the text at both the warning-log and re-raise sites; redaction runs before the message is truncated so a secret straddling the boundary is still masked. The exception type and retry behavior are unchanged.
The client retry loop retried every provider exception uniformly, so an HTTP 400/401/403/404/422 (auth or malformed-request failure that will never succeed) burned the full retry budget with backoff sleeps before surfacing. A new _is_retryable predicate classifies failures -- retryable: 429, 5xx, 408, 425, and connection/timeout errors; non-retryable: 4xx auth/validation -- so non-retryable errors raise immediately (retryable=False) while retryable ones keep the existing retry+backoff behavior. Provider-error redaction and the final raise type are unchanged.
…fecycle (#60) A failing event destination in ThreadSafeEventEmitter._drain starved every later destination of the same event; the drain now isolates each destination (catches its failure, logs with traceback, counts it, and continues) so one broken sink cannot drop events for the rest. The redis stream path re-raises RedisError instead of silently swallowing it so the isolation guard records the failure. stream_from_worker gains an opt-in bounded queue with drop-oldest overflow (a stalled consumer can no longer block the producer forever), an opt-in max lifetime, and deterministic worker-task cancellation-and-await in the finally block so no coroutine outlives the generator.
_attach_to_matching_family scanned families in undefined DB order and matched family names as bare substrings, so the same fingerprint could attach to different families across runs and a short name like 'aes' matched inside unrelated words such as 'cases'. The scan is now ordered by created_at ascending and bounded to 500 candidates, and matching uses a token-boundary check (summary tokenized on ASCII alphanumerics) with a three-character minimum name length, extracted into a pure _family_name_matches helper.
build_writeup assembled an unbounded user bundle and returned the raw LLM output, so a disk-image case with thousands of artefacts could blow the context window and emit an arbitrarily large document. The bundle now refuses (ValueError with a per-section byte breakdown) when it exceeds four times the section cap, the LLM call passes max_output_tokens so generation is bounded at the model layer (LLMClient.chat gained an opt-in per-call ceiling that only narrows the routing cap), and the returned content is post-truncated to 64000 characters with an explicit marker.
VRFindingRecord.evidence_refs_json was written and read as untyped JSON, so a mis-typed ref was silently rendered as an empty section in the PDF report. Writes now route through a discriminated-union EvidenceRefList (source_citation and poc_draft_metadata), which raises at write time on an unknown kind or forbidden field; a before-validator normalizes legacy bare-string source ids so existing rows still validate on read. The pdf_report consumer reads through the same validator. No schema change -- the column stays JSON text; only its contents are now validated.
… C2) A change to a secret-classed config key wrote the old and new plaintext into the config_security_change PlatformEvent (which fans out to the immutable audit trail) and into the PUT /config audit event details. Once written, a secret in the audit chain cannot be purged without breaking the hash chain. ConfigRegistry.set now redacts both values in the emitted event and records a sha256 of the old->new transition so a rotation stays auditable without persisting the secret; update_config_value redacts the audit value and records was_secret. Non-secret keys keep cleartext.
delete_project removed investigations, agent steps, writeups, artifacts, leads, and project evidence but left answer candidates, analyst directives, finding suppressions, and solid-evidence rows orphaned in the database. All four are now deleted in the same transaction -- answer candidates by both investigation_id and project_id (they can be linked either way), the rest by project_id. The C2 append-journal record of the deletion and the ON DELETE CASCADE migration remain follow-on scope.
New substrate the audit trail (#52), observability (#39), graph journal (#23), replay corpus (#32), and untrusted-execution evidence (#58) build on. platform_journal is an append-only, hash-chained event log keyed by (chain_id, seq) where chain_id is team:{team_id} or global; each row carries row_hash (chains to the previous row) and payload_hash (covers the possibly-redacted payload independently). Migration 071 adds the table, a dead-letter table, key indexes, and a BEFORE UPDATE OR DELETE trigger enforcing immutability. journal.append shares the caller's transaction, fails closed (JournalWriteError), resolves team from session.info team_context, applies C6 redaction to secret-classed payload keys before hashing, and allocates seq Python-side with a chain-head read plus PK-collision retry (the mirror of workflows/log.py). journal.verify_chain recomputes the chain to detect any rewritten or reordered row; append_or_deadletter is the legacy-only fallback. The Postgres stored-function / FOR UPDATE NOWAIT hot-chain optimization is a documented follow-on.
…st (#52) BoundedEvidencePack was a mutable bucket with a self-reported drop list and no way for a consumer to confirm the rendered contents matched what was added. It now seals: seal() computes a merkle root over the ordered per-section hashes and freezes the pack (add() raises EvidencePackSealedError afterward); verify() recomputes the root and detects any post-seal content change or reordering. seal_and_journal appends one evidence_added row per section (content referenced by content_hash, not stored inline) plus a final evidence_sealed row carrying the digest into the C2 platform journal, so the seal is itself hash-chained and tamper-evident. The EvidenceSection frozen-model refactor (making char_count a computed field) remains follow-on hardening; verify() already provides the tamper-evidence guarantee.
Add a synchronous journal append path and route the CLI audit callsites through it, so admin CLI actions land in the hash-chained platform journal rather than the flat, rewritable AuditEventRecord table. - journal.append_sync: sync-session counterpart to append, same fail-closed hash-chain semantics (seq via MAX(seq)+1 in a savepoint, IntegrityError retry, full 64-hex row/payload hashes). Shared preamble and row build factored into _prepare/_build_row so the async and sync paths cannot drift. - audit.record_audit_event_sync: journal-backed sync wrapper mirroring record_audit_event's signature; maps user_id to actor and stashes target/details in the payload. Fail-closed so a broken chain aborts the caller's transaction instead of dropping the row. - cli.py: the three audit callsites (direct-tool run success/error, API key creation) now call record_audit_event_sync. The legacy async path keeps writing AuditEventRecord during the rollout. Tests: append_sync chains and passes the async verifier; the CLI wrapper writes a kind=audit journal row with the expected actor and payload.
A single hung tool (e.g. an audit-mcp cold build) could block the entire
LLM turn indefinitely: the tool-calling loop awaited tool_executor with no
time bound.
- LLMRouting gains tool_timeout_s (default 300s), resolved per task_type via
llm_tool_timeout_s_{task_type} then llm_tool_timeout_s, both falling back
to 300s.
- The tool-calling loop now runs each tool_executor call under
asyncio.wait_for(timeout=tool_timeout_s). On timeout the model receives a
synthetic tool result {error: tool_timeout, tool, timeout_s} and the loop
continues so the model can react. The LLM call is not retried from scratch;
the timeout is treated as a domain-level tool failure.
Tests: a 60s tool bounded at 0.1s returns a tool_timeout result and the loop
finishes in under 5s; resolver precedence (task-specific > global > 300s
default) and non-numeric fallback.
_call_with_retry built a fresh AsyncOpenAI on every call and never closed it; each client owns an httpx connection pool, so the file-descriptor count grew unbounded under load. - Add a process-local _AsyncOpenAIPool keyed by (api_key, base_url, timeout). Construction is synchronous, so get() has no await point and needs no lock on a single event loop. Routing is frozen per investigation, so the pool converges to a small set of long-lived clients whose connections are reused. - Route both creation sites (_call_with_retry and _inner_call) through the pool. _inner_call no longer closes its client per call, since the pooled client is shared. Graceful close of pooled clients on platform shutdown is a follow-on (needs lifespan wiring); the dedup here is what bounds the descriptor growth. Tests: the pool returns the same client for an identical key and distinct clients when api_key, base_url, or timeout differ.
The LLM config resolvers are async (await registry.get / secret_store resolve_provider_secret), but the client test suite's FakeRegistry and FakeSecretStore still exposed sync methods, so 19 tests errored with "object NoneType can't be used in 'await' expression" and gave the client retry/tool/routing paths no coverage. - Make FakeRegistry.get and FakeSecretStore.resolve_provider_secret async. - Drop the duplicate _AsyncFakeRegistry helper (FakeRegistry now serves the tool-timeout resolver tests directly). - Lint the restored file: rename the mixed-case MockOAI patch handle to mock_oai and hoist the openai APIConnectionError/APITimeoutError imports to module scope. Test-only; no production change. tests/platform/llm/test_client.py: 19 failed + 12 passed becomes 31 passed.
The LLMConfigProvider resolvers are async, but test_config.py still called them synchronously against sync fakes, so 24 of 25 tests errored and the routing resolution logic had no coverage. - Make FakeRegistry.get and FakeSecretStore.resolve_provider_secret async; every test is now async and awaits the resolver under test. - Correct two stale expectations: the hardcoded model fallback is antigravity/claude-opus-4-6-thinking, not the old openai/gpt-4o-mini. Test-only; no production change. tests/platform/llm/test_config.py: 24 failed + 1 passed becomes 25 passed.
The AsyncOpenAI pool added _client_pool to AilaLLMClient.__init__, but _make_stub_client builds the client via __new__ to isolate the retry loop and so bypasses __init__. _call_with_retry now reads self._client_pool, which the stub did not set, so the five loop-level tests raised AttributeError. Set _client_pool on the stub to mirror __init__.
Six suites under tests/platform/llm/ errored because the config/pipeline layer became async but the test fakes stayed sync, and a handful of expectations lagged deliberate production changes. Restored to green with no production edits; each expectation change was verified against current source. - Async fakes: FakeRegistry.get and FakeSecretStore.resolve_provider_secret are now async everywhere the resolvers await them; affected tests await the resolver under test. test_cost keeps a sync get_sync twin for CostTracker._resolve_ceiling. - Fail-mode defaults: resolve_fail_mode now returns "closed" by default for security-critical steps (sanitize/validate/gate/verify/classify/seal, per the §156 _SECURITY_CRITICAL_STEPS set). Tests that assumed an "open" default for classify/validate now seed an explicit "open" to exercise the fail-open branch, matching how an operator would opt in. - test_validate: the CVE lookup mock target moved from a removed session_scope to VulnEvidenceValidator._batch_lookup (ServiceFactory().storage.fetch_all). - test_cost: swallow assertions now expect sqlalchemy SQLAlchemyError, which is what the narrow production except actually catches; integration tests mock async_session_scope and use per-invocation run ids so real-DB state cannot leak across runs. - Lint: hoisted deferred imports to module scope and renamed mixed-case patch handles to satisfy ruff. Counts: test_classify 19 failed -> 76 passed; test_gate -> 41 passed; test_pipeline 22 failed -> 30 passed; test_validate 22 failed -> 40 passed; test_cost 9 failed -> 37 passed; test_integration async-fake fix (network tests remain deselected by default). No production bugs surfaced.
- Async fakes for the config provider (get / resolve_provider_secret). - POST_CALL_STEPS gained "verify" (pipeline.py:35); update the assertion. - is_step_enabled / resolve_fail_mode are async; the two config smoke tests now await them. classify defaults to fail-closed as a security-critical step (config.py _SECURITY_CRITICAL_STEPS), so the fail-mode assertion expects "closed". Test-only. tests/platform/llm/test_pipeline_smoke.py: 3 failed -> 7 passed.
_INJECTION_PATTERNS is a dict keyed by name (idempotent registry), but the test_custom_pattern_applied teardown still called list-style _INJECTION_PATTERNS.pop() with no argument, raising TypeError. Pop by the registered key instead. Also hoist the deferred per-test imports (LLMResponse/_enrich_response, ScoringCandidate, build_signal_payload) to module scope for ruff, keeping the one intentional in-test import in test_sanitize_importable_from_llm_package (which asserts package-level importability) under a noqa. Test-only. tests/platform/llm/test_sanitize.py: 1 failed -> 47 passed.
Scheduled report emails verified the server certificate against the system
trust store over STARTTLS only. Two gaps remained for locked-down mail
infrastructure: no way to trust a private CA, and no implicit-TLS (SMTPS)
support for servers that require port 465.
- New platform config keys: smtp_ca_bundle_path (path to an admin-managed CA
bundle; system trust store when unset) and smtp_use_implicit_tls
("true" -> SMTPS on 465 instead of STARTTLS).
- _send_report_email builds the TLS context with the CA bundle when set (an
invalid path fails loudly rather than downgrading trust) and connects via
smtplib.SMTP_SSL for implicit TLS, keeping certificate verification on in
both modes.
Tests: implicit-TLS uses SMTP_SSL with a verifying context and never touches
plaintext SMTP/STARTTLS; the CA bundle path reaches the TLS context.
persist_cost_record called check_monthly_budget inside the same try that only caught SQLAlchemyError, so a leaked budget-alert exception (anything other than a DB error) propagated out even though the cost record had already committed -- letting best-effort budget alerting fail durable cost recording. Move the budget check out of the DB-write try into its own guard that runs only after a successful commit and swallows the realistic leak set (SQLAlchemyError, RuntimeError, ValueError, TypeError, OSError) with a warning. The cost record is preserved regardless of budget-check outcome.
estimate_human_cost documents (behavior 7) that an LLM call failure is logged and returns None, but its except caught only AILAError / ValidationError / SQLAlchemyError. LLMError is a standalone Exception, not an AILAError subclass, so a failed chat_structured call propagated out of estimate_human_cost and up the caller instead of degrading to None. Add LLMError to the handled set.
check_monthly_budget documents "Never raises -- all exceptions are logged and swallowed" so budget alerting can never block cost recording, but its except caught only SQLAlchemyError. A registry connection error (ConnectionError / OSError) or a session-scope RuntimeError propagated out, breaking the fire-and-forget contract its callers rely on. Swallow the realistic leak set (SQLAlchemyError, RuntimeError, ValueError, TypeError, OSError) with a logged warning.
The 5-step cost recording block runs after a successful provider call, but its steps caught only narrow types (SQLAlchemyError / ValueError). A leaked RuntimeError, OSError or similar from calculate_cost_usd, persist_cost_record, the Prometheus counter, or the domain-event publish escaped to the outer retry loop, which treated it as a retryable provider error and, after exhausting retries, raised LLMError -- turning a good response into a failure. Introduce _COST_TELEMETRY_ERRORS (the realistic best-effort leak set) and apply it to all five steps so each swallows its own failure independently and the LLM response is always returned. Provider-call retry classification is unchanged; only the post-success telemetry steps are broadened.
…nt (#62) The C2 platform_journal CheckConstraint used char_length(), a Postgres-only function. SQLModel.metadata.create_all against a SQLite engine (used by a range of fast unit tests) then failed with "no such function: char_length" while building the table, erroring out every test in those modules before their own assertions ran. length() is equivalent for text on Postgres (length(text) == char_length(text)) and exists on SQLite, so the constraint now builds on both. The enforced invariant (row_hash / payload_hash are exactly 64 hex chars) is unchanged; journal tests still pass against the Postgres test DB. Migration 071 is aligned to length() so a fresh migrate matches create_all.
…tgres (#62) ConfigRegistry.register/set/get and db_upsert/db_delete became async on async_session_scope, but these suites still called them synchronously against an in-memory SQLite engine with a monkeypatched sync session_scope, so every test errored (unawaited coroutine) or failed to build tables. - Add a storage_db fixture (tests/storage/conftest.py): creates the full schema in the Postgres test DB and truncates every table per test -- the storage-suite analogue of the api test_db (D-48/D-49: no SQLite). Hoist the conftest's deferred imports to module scope so it is ruff-clean. - test_config_audit: async tests awaiting register/set/get against the Postgres test DB; register persists defaults without emitting, so the per-test event count still reflects only set() calls. - test_operations: db_upsert/db_delete tests are async and use an AsyncSession from storage_db; the sync cached_fetch tests are unchanged. tests/storage/: 17 failed + 25 passed becomes 42 passed. Test-only.
test_seal used sync fakes and an in-memory SQLite engine with a monkeypatched sync session_scope, but ConfigRegistry is async and make_seal_step persists via async_session_scope -- so seal.py awaited a sync fake (str) and the SQLite engine was never written to. Every DB-backed test failed. - FakeConfigRegistry.get/set and FakeConfigProvider.is_step_enabled/ resolve_fail_mode are now async; every DB test is async and awaits the step. - Drop the SQLite in_memory_engine / _patch_session_scope fixtures; DB tests depend on the Postgres test_db and query AuditSealRecord via async_session_scope where the step actually writes. - Correct one stale expectation: when content storage is enabled with an HMAC key, seal.py encrypts on write (LLM-SEC-03), so the plaintext columns are cleared and the encrypted columns hold the content. The test now asserts the encrypted columns. - Hoist the deferred PipelineRunner import for ruff. tests/platform/llm/test_seal.py: 15 failed + 5 passed becomes 20 passed; the full tests/platform/llm/ directory is now green (499 passed).
…cts (#62) Five tests lagged deliberate CyberReasoningEngine changes: - decide_next_turn now calls chat_structured (not chat); the fake LLM client lacked that method. Added it, recording calls the same way. - tool_run command is now a JSON dispatch object {tool, args} enforced by ReasoningTurnDecision._validate_tool_run_command; the fixture used a raw shell string. Use a JSON command and assert it is kept as-is. - absorb merges live hypotheses across turns (nothing the agent proposed vanishes unless explicitly rejected), so the merge keeps H1 alongside new H2 rather than replacing with just H2. - render_case_model now labels the populated hypothesis section with its live count ("Live hypotheses (1):"), renders tool readings with the key and body on separate lines, and headers the agent-scratchpad bucket with its total count -- assertions updated to match, preserving the partition intent. Test-only. tests/platform/test_reasoning_engine.py: 5 failed becomes 11 passed; tests/platform/ (excluding llm) is now green.
) VulnerabilityTool.__init__ no longer calls init_db (schema bootstrap moved to platform startup), and SingleActionTool.forward/_execute became async. The suite patched the removed _base.init_db symbol (AttributeError on every test) and called forward/_execute synchronously (coroutine, no raise/subscript). - Drop the init_db patches and assertions; delete the init_db-specific test. - Make the concrete _execute async and await forward/_execute in the SingleActionTool tests. Test-only. tests/modules/vulnerability/tools/test_base.py: 10 failed becomes 9 passed.
First increment of hot-pluggable MCP: an operator-editable catalog of server instances, without touching the live dispatch path. - instance_catalog.py: McpServerInstance table (mcp_server_instances, migration 092) + McpInstanceCatalog async CRUD service. capability_tags stored as JSON text; natural key (module_scope, name) so one server name can appear under multiple modules. - routers/mcp_instances.py: god-tier-admin CRUD at /platform/mcp/instances (GET/POST/PATCH/DELETE), 409 on duplicate scope+name, 400 on bad transport / empty patch, 404 on unknown id. - registry.py: McpRegistryServiceBase URL resolution gains a catalog tier -- env > ConfigRegistry > enabled catalog row > code default. A disabled row is a no-override; a catalog DB error logs at WARNING and falls back to the default. When no row exists for (module_scope, name), resolution is byte-identical to the prior behaviour. The catalog starts empty (no seed), so live dispatch is unchanged until an operator adds rows; bridges, adapters, the tool executor, and _applicable_servers_for_kind are byte-untouched. Tests: 25 (catalog model round-trip + service CRUD, JSON tag codec, scope filter; API create/list/patch/delete/duplicate/non-admin).
…ase (RFC-03 Phase 7) The per-branch reasoning turn (vr 502L, malware 549L, 0.86 normalized similarity) is consolidated into AgentTurnRunnerBase.run_turn on the platform, the sibling of the already-merged ToolExecutorHelpersBase. Both researchers now inherit one turn loop. Per-module differences are: - config class attrs: _LOG_LABEL, _error_cls, _result_cls, _message_model, _branch_model, _OUTCOME_STATE_APPROVED, _EMPTY_TOOLRUN_DIRECTIVE. - staticmethod bindings for the per-module module-level helpers (_fetch_tool_specs, _load_prompt, _decision_to_message_payload, _terminal_outcome_kind, _outcome_payload, _upsert_canonical_outcome, _resolve_task_type, _evaluate_quorum, _upsert_review), bound at module import time since they are defined below the class. - override hooks: _extra_user_prompt_kwargs (VR adds cve_intel), _maybe_reject_fanout_submit (VR variant-hunt gate / malware fan-out gate), _review_vote_and_comment (malware downgrades an empty-rationale reject to abstain), _dispatch_approved_outcome (per-module dispatch task), _handle_edit_outcome (malware edit_outcome action). The shared skeleton (load, prompt build, idempotency-cached engine call, absorb, message + branch persist, terminal outcome upsert, review handling, empty-tool_run coerce) now lives once; both modules keep only their domain helpers and the hook bodies. Removes ~500 duplicated lines. Verified: compileall, ruff, honesty_audit (0 findings), and a full faithfulness audit confirming every vr/malware run_turn divergence maps to a hook, config attr, or staticmethod binding while the shared non-seam skeleton stays line-identical. Structural smoke asserts both researchers inherit the base run_turn with every binding/attr/hook resolving and the correct per-module overrides. 125 agent + loop + researcher tests stay green. run_turn drives the live DB + engine + MCP and has no unit-test surface; its end-to-end behavior is gated on the operator live-investigation smoke at merge.
Session progress (2) -- RFC-03 run_turn capstone + ADL/MCP catalog (dev
|
… (RFC-03) Closes the RFC-03 config-drift acceptance item: no os.environ / os.getenv under modules/*/agents/**. Six import-time env-read caps are replaced by operator-tunable ConfigRegistry keys resolved at the use site (env -> DB -> schema default), so a PUT /config override lands on the next turn without a worker restart. - Register keys: VR (variant_hunt_reject_cap, unresolved_hyp_reject_cap, tool_executor_hard_block_repeat) and malware (unresolved_hyp_reject_cap, sub_investigation_reject_cap, max_sub_inv_per_parent) in the module ConfigSchemas, all with their prior defaults (3, except max_sub_inv=5). - AgentTurnRunnerBase gains a no-op _load_turn_config hook called once at the top of run_turn; each researcher overrides it to stash its gate caps as instance attributes via get_int, so the sync submit-gate methods read a resolved value without an await. Drops the module-level cap constants. - ToolExecutorHelpersBase.execute reads the hard-block threshold via a new _hard_block_repeat_limit hook (base default None; VR resolves from config); malware keeps the None default (no hard block). - The malware outcome dispatcher reads max_sub_inv_per_parent via get_int inside _dispatch_analysis_report. Behavior note: the operator env-var names change from the raw VR_* / MALWARE_* form to the standard AILA_VR_<KEY> / AILA_MALWARE_<KEY> form (the same migration the cap-exceeded reaper caps already made). Defaults are unchanged, so an operator who never set the old env vars sees no difference. Verified: honesty_audit 0, the acceptance grep returns nothing, schema keys match the get_int calls, 130 agent + researcher + dispatcher + loop tests green.
RFC program complete -- dev at
|
…C-05)
The shared search router hardcoded module_id="vulnerability" on finding
results even though the module was resolved by capability via
first_with("latest_findings"). A second module exposing latest_findings
would have had its results mislabeled as vulnerability. The value now
reads module.module_id from the resolved module. The platform_names_module
honesty rule checks .require() calls, so this string literal had evaded
the audit.
…o config (RFC-12) Closes the provenance half of RFC-12 criterion 1 and the pattern relevance-floor config-schema wiring. - KnowledgeEntryRecord gains model_id, content_hash, source_type (indexed) and updated_at columns (migration 093, all nullable so the add runs on a populated table with no backfill). KnowledgeService.store now stamps every write -- model_id from the resolved embedding provider, content_hash as the sha256 of the stored text (per chunk on the chunked path), source_type from the ingest kind, updated_at on insert and update. This makes each stored vector attributable to the model that produced it and detectable on content change. - PlatformConfigSchema gains knowledge_pattern_relevance_floor (default 0.3). PatternStoreBase._resolve_relevance_floor now resolves it from the platform config schema via ConfigRegistry, keeping the module constant as the last-resort fallback. The stale "a future schema field" comment is removed now that the field exists. Contextual enrichment, adaptive routing, CAG/graph, sanitize/classify, and record-replay retrieval eval remain later RFC-12 increments.
…RFC-02) Closes RFC-02 criterion 2. build_branch_summary, build_message_summary, and build_outcome_summary now live in platform/services/investigation_ summaries.py alongside build_investigation_summary, and both vr and malware api_routers call them instead of carrying private copies that could drift. Divergences preserved, not dropped: the malware branch summary keeps its acked_at field (the platform builder forwards it only when the summary contract declares it), and the outcome builder takes a review_counts dict so malware's five vote-count fields survive while VR keeps its zero-count default. The NULL-state dispatch fallback is now shared by both modules.
…-10)
Closes the "without a code release" operator surface of RFC-10 criterion 2.
AgentLifecycleController.evaluate/promote/rollback/list_transitions are
exposed at POST /admin/lifecycle/{evaluate,promote,rollback} and GET
/admin/lifecycle/transitions, god-tier-admin gated, rate-limited, and
DataEnvelope-wrapped, mirroring the admin_eval router. The actor is derived
from the auth context, not the request body, so it cannot be forged.
StageTransitionError surfaces as HTTP 409 (a promote with no passing
evaluate journal row is rejected), BenchmarkNotFoundError as 404. No
schema change: lifecycle_transitions already shipped in migration 091.
Shadow/canary stages and the review-quorum promotion gate remain tracked
follow-ups.
…-10) Closes the "eval gate AND quorum approval" half of RFC-10 criterion 1 and RFC-08 criterion 2. Promotion was gated on eval-pass alone; a distinct- approver quorum now stands beside it. - LifecycleStage gains APPROVED (between EVALUATED and PRODUCTION; no migration -- transition stages are TEXT). - AgentLifecycleController.approve(key, version, actor, reason) requires a passing evaluated journal row, then journals evaluated -> approved. - promote() now also counts DISTINCT approver actors on approved transitions for (key, version) and refuses with StageTransitionError naming approvals-present vs required when below the quorum. The promote journal snapshot records approver_count + quorum_threshold. - PlatformConfigSchema.agent_promotion_quorum (default 1) is the required distinct-approver count, tunable via PUT /config with no restart. - POST /admin/lifecycle/approve exposes it; actor derives from the auth context. The eval-runner auto_promote fast path stays eval-only and admin-opt-in by construction; the quorum-gated path is the controller.
…patcher to platform (RFC-03 Phase 5/6) Removes three duplicated agent primitives that the audit found still lived as full copies in both vr and malware -- the exact duplication RFC-03 exists to eliminate. Each is now a platform base with per-module hooks and class attributes; the module classes keep their names as thin subclasses, so every existing import site and aggregator re-export stays untouched. - PersonaRouter: the shared persona -> role table and coercion move to the platform; each module supplies its default_task_type plus its role- or persona-keyed task_type table as class attrs. - PatternExtractor: the ~550-line body collapses to an 85-line subclass per module; PatternExtractorBase owns the load -> gate -> transcript -> prompt -> idempotent_llm_call -> parse -> store flow, with record models, enums, prompt path, and task_type as class attrs. - OutcomeDispatcherBase owns the claim -> guard -> per-kind route -> persist skeleton (the TOCTOU stays closed via the platform claim_outcome_for_dispatch service); each module supplies its per-kind handlers, state guard, and terminal persistence (VR keeps its sibling- halt + investigation-complete + ARQ-purge cascade). Behavior note: VR's dispatch of a missing outcome now returns a SKIPPED outcome_not_found result instead of raising ValueError, converging on malware's shape. Verified inert: two VR callers wrap the call and two dispatch an outcome read + committed moments earlier, so the not-found path cannot fire. Characterization tests pin both module configurations for each primitive; 191 agent + lifecycle tests pass.
…form (RFC-03 Phase 5) Removes the last two duplicated agent primitives the audit found living as full copies in both vr and malware, completing RFC-03 Phase 5/6. - ClaimVerifierAgentBase owns the two-stage extractor -> verdict pipeline, the probe allowlist + signature fetch + payload render, the parsers and verdict renderer, the persist unit-of-work, and the auto-promote / revert skeleton. vr collapses 1140 -> 149 lines, malware 1212 -> 238. Per-module knobs (task_type keys, negative-phrase tables, record models, outcome-dispatcher class, promote source/target kinds, auto-promote floor, claim-text extraction) are class attrs + hooks. Both idempotent LLM call sites and the OutcomeDispatcher promotion call are preserved. - SynthesisRunnerBase owns the load -> panel -> prompt -> chat_structured -> validate -> persist flow with the two-UoW alive-status re-check and the terminal status flip. vr and malware supply their own SynthesisResponse schema, system prompt, and panel renderer; malware keeps its richer schema + SynthesisOptions via hook overrides (force-resynthesize, status-flip suppression, payload extras). Prompts stay inline as class attrs for now; routing them through the version store is the separate RFC-09 slice. Characterization tests pin both module configurations at the mocked idempotent-LLM seam; 131 new + 40 existing synthesis/claim tests pass.
… (RFC-09) Extends the prompt-attribution recording from the single researcher-turn path to every agent LLM call, and adds the missing prompt_version field. - The correlation context gains prompt_version alongside prompt_content_hash, with a current_prompt_version accessor. LLMCostRecord and AuditSealRecord gain a prompt_version column (migration 094); the cost + seal writers stamp it. - idempotent_llm_call now opens a correlation scope around the model call with set-if-unset precedence: an explicit arg wins, then an outer scope (the researcher turn), then the sha256 of the system prompt. Every call routed through it (claim verifier, pattern extractor, synthesis, nday, narrative) therefore lands a content_hash on its cost + seal records instead of leaving them null. Callers may pass prompt_version to attribute a resolved version. This closes the recording half of RFC-09 criterion 2 for the agent call path. Routing the remaining report/workflow inline prompts through the registry, pin-per-investigation, and the model-family key are the next RFC-09 slices.
Closes RFC-09 criterion 4 -- a live production-alias flip no longer re-routes a running investigation. InvestigationRecordBase gains prompt_pins_json (migration 095, on vr_investigations + malware_ investigations), a JSON map of prompt-key -> resolved version. resolve_pinned_prompt (new platform/prompts/pinning.py) implements lazy pinning: the first resolve of a key for an investigation records the current production version into the pin and uses it; every later resolve reads the pinned version and resolves that exact version instead of the live alias. Fail-open is preserved -- a store error or a missing investigation falls back to the file registry, and no pin is written when there is no production alias. Both researchers' _load_prompt return a LoadedPrompt(body, version); turn_runner threads investigation_id in and stamps the resolved version into the correlation scope so the cost + seal writers record the exact version (closing the version half of criterion 2 on the researcher path). Model-family key variants (criterion 6) are deferred: the routing layer exposes no first-class model family to key on. Pin tests cover first- resolve pinning, alias-flip inertness, per-investigation isolation, and store-error fallback.
… (RFC-09) Closes RFC-09 criterion 1 for the six LLM call sites that do not go through idempotent_llm_call: the human-cost estimator, the VR section writer, the VR advisory-narrative and PoC-development workflow states, and the forensics writeup builder + free-flow investigator. Each inline system -prompt literal moves to a .md file resolved through the file-backed PromptRegistry (byte-identical to the prior text, verified, so no cost or seal drift), and each LLM call is wrapped in a correlation scope that stamps sha256(prompt) while preserving any outer investigation/branch/turn/ prompt_version attribution. The forensics free-flow prompt keeps its OS- hint assembly in code over base + per-OS .md fragments, hashing the final assembled prompt. These sites now record content_hash on their cost + seal records instead of leaving them null. The agent-reasoning prompts still held as class attrs on the platform ClaimVerifier/Synthesis/PatternExtractor bases route through idempotent_llm_call (already content-hash-stamped) and remain a criterion-1 follow-up.
Adds two honesty-audit rules that lock in the RFC-03 agent-runtime consolidation so a future edit cannot silently re-duplicate what was lifted to the platform. - Rule 42 (agent_primitive_reimplementation) extends its lifted-primitive set with run_turn and the four turn helpers (decode_case_state, encode_case_state, auto_resolve_live_on_terminal, to_outcome_confidence) and now walks one level of class-body method defs, so a module subclass that overrides run_turn is flagged while a thin subclass and import re-exports stay clean. - Rule 49 (agent_env_read, new) fires on os.environ / os.getenv access under modules/*/agents/**, locking in the config-drift closure that moved those caps to ConfigRegistry. Both are zero-finding-clean on the current tree. Guardrails that would false-positive on intentionally-kept code (the inline base-class prompts, the static MCP catalog + bespoke bridges) or need unshipped infra are deferred with reasons rather than added unsafely.
…-04/05) Makes _template demonstrate the platform primitives so a module copied from it starts boundary-clean. Adds TemplateConfigSchema subclassing ModuleConfigBase (extra='forbid' posture, example fields), declares the ModuleProtocol registry methods (reasoning_strategies, reasoning_domain_profiles, workflow_definitions) as documented typed stubs using only public platform imports, registers the config schema in register_tools mirroring the real modules, and turns services/__init__ into a pointer at the platform investigation-support bases. The investigation-agent engine stays out of _template by design -- it is a simple-action scaffold, and the reusable agent template is the platform bases plus the two reference modules.
Adds an Unreleased section covering RFC-01 through RFC-12 and the faithfulness-remediation pass: the platform agent runtime and primitive consolidation, prompt versioning + deployment + per-investigation pinning, the eval-gated agent lifecycle with a review quorum, the MCP instance catalog, knowledge provenance, self-healing, and the honesty guardrails that lock it in. Flags the two operator-facing contract changes (the AILA_VR_/AILA_MALWARE_ env-var rename and the promotion-quorum gate). The version number is set at the dev-to-main release.
…tes (RFC-07)
Closes RFC-07 acceptance bullet 2. The recovery-service duplication the
bullet targets was already removed by RFC-04 (the module recovery files
are functools.partial bindings over platform functions), and step 0's
fail-closed conversions + SSE-failure counter already shipped; the genuine
gap was the single ResilienceLayer the Design sketches.
New platform/services/resilience.py exposes one facade -- classify_failure
(delegating to InfraDeathClassifier), should_retry, conservative_default,
and record_signal -- with a RecoveryPolicy and a FailureVerdict. A new
aila_resilience_signals_total{op,source} umbrella counter funnels every
fail-closed signal; record_signal also mirrors the existing
aila_sse_write_failures_total so operator dashboards read unchanged. The
queue defer, SSE emitter, workflow-log emit, and finalizer classifier
sites now route through the layer instead of each carrying its own
log-then-metric-then-return. The verify + pipeline steps already re-raise
(fail-closed) and are unchanged. 35 layer tests plus the existing
fail-closed + infra-death tests pass.
…(RFC-08) Closes RFC-08 criteria 3-5, the self-improvement loop that sits behind the already-shipped eval gate. All three are platform services generic over injected record types (no module import). - ExperienceWriter turns a reviewed outcome into a signed pattern in the PatternStore: a positive pattern on an accept verdict, a negative one on a reject so retrieval can down-weight a rejected approach. It reuses the PatternStore create path rather than duplicating signing. - CalibrationProposer aggregates accept/reject history per outcome_kind into a versioned, reversible confidence-threshold proposal (eval_calibration_proposals, migration 097; status active/superseded/ reverted). Proposals are advisory -- application still goes through the eval gate and the lifecycle, never auto-applied. - RoutingLearner turns outcome + cost history into a typed routing recommendation. The pre-execution sizing consumer (#23) does not yet exist, so the recommendation is exposed and published as recommendation-only rather than wired to a nonexistent seam. Real aggregation over real history rows; tests cover positive/negative pattern writes, a versioned+reversible proposal, and a recommendation.
Closes RFC-10 criterion 2's shadow/canary path and criterion 5's hold/alert. The controller gains shadow(), canary(cohort_percent), promote_from_canary() (still eval + quorum gated), record_canary_signal(), and the resolvers active_shadow / active_canary / resolve_version_for_investigation. A shadow registers a candidate for comparison without production traffic; a canary routes a stable cohort fraction of NEW investigations to the candidate by a SHA-256 bucket of the investigation id; a drift or cost sample past the configured ceiling flips the active canary to a HELD state and raises a signal. Assignments live in lifecycle_canary_assignments (migration 096); ceilings are agent_canary_drift_ceiling + agent_canary_cost_ceiling_usd on PlatformConfigSchema. admin_lifecycle gains POST /shadow, /canary, /canary/signal and GET /route. 16 tests cover registration, stable cohort bucketing, the hold-on-spike path, and the gated promote-from-canary.
…RFC-11) Advances RFC-11 through steps 0/3/4 plus call provenance. A new generic McpClient owns the transport that was duplicated inside the three bridges; each bridge now resolves its endpoint through the shared resolve_instance (preserving the env>config>catalog>default precedence and the byte- identical empty-catalog fallback) and keeps only its genuinely server- specific request/response shaping, which RFC-11 section 6 permits as plugin-side rendering. The registry gains resolve_by_capability, bind, and pool_for_capability; the researchers resolve servers by declared capability tag from the catalog (falling through to the name map when the catalog is empty) instead of a hardcoded kind->server list, and an InstancePool round-robins across enabled rows of one capability. Every call records the serving instance_id (new column on the shared McpCallLogRecordBase, migration 098, on both module call-log tables). The three bespoke bridge classes are not deleted: after transport unification their remaining bodies are legitimate server-specific adapters (JADX rewrite, IDA name coercion, APK path resolution), not duplicated transport. 23 client tests plus the catalog + bridge tests pass.
…togen Adds the bottom-of-file side-effect imports for CalibrationProposalRecord (RFC-08, eval_calibration_proposals) and LifecycleCanaryAssignment (RFC-10, lifecycle_canary_assignments) so SQLModel.metadata sees both tables on the create_all path and Alembic autogen, matching migrations 097 and 096.
Closes RFC-12 criterion 7. A retrieval benchmark records queries with per-query ground-truth relevant ids; RetrievalEvalRunner.run replays each query through an injected retrieve function, scores the ranked ids with precision@k, recall@k, MRR, nDCG@k, and MAP, and applies a beats() gate so a retrieval change is eval-gated the way prompt changes are. Two tables back it (retrieval_eval_benchmarks + retrieval_eval_runs, migration 099), mirroring the prompt-eval harness. The runner takes the retrieve function by injection, so the platform eval layer does not import a module. Real metric math, verified against hand-built ranked lists; 30 tests.
…t gate (RFC-12) Closes RFC-12 criteria 4, 5, and 6 on the retrieval path. A KnowledgeRouter classifies a query to one of three paths: stable-core (a preloaded cache of designated entries returned without a vector search), simple (the existing hybrid vector + FTS + relevance floor, unchanged so current callers are unaffected), or graph (multi-hop). The graph path adds a knowledge_entry_edges table (migration 100) and a bounded BFS traversal that follows labelled edges from a seed match with a hop cap. Every returned result passes a sanitize/classify gate and carries its provenance (model_id, content_hash, source_type, updated_at). retrieve_routed exposes the router; the agent-facing KnowledgeRetrieveTool consults it. 26 tests.
Adds the side-effect imports for RetrievalBenchmarkRecord / RetrievalRunRecord (RFC-12 eval, migration 099) and KnowledgeEntryEdge (RFC-12 graph, migration 100). The knowledge-edge import lands at the end of the module, after every record class is defined, so the transitive db_models references in the knowledge-graph import chain resolve without a circular import.
Closes the enrichment half of RFC-12 criterion 2. KnowledgeService.store gains an opt-in enrich flag (default off, so the current path stays byte- identical); when on, each chunk gets a short LLM-written blurb situating it in its parent document, prepended before embedding so the vector and the FTS search text both carry the context. The blurb prompt is a registry .md file (no inline literal), the call routes through idempotent_llm_call, and entry_metadata records the blurb + the original chunk text so the raw chunk stays recoverable. No schema change -- metadata rides the existing entry_metadata JSON. The enrich_chunk import is deferred to the call site to avoid a module-load import cycle. 4 tests cover the on/off paths.
Closes the RFC-02 Class-A cursor-keying defect. Cursor rows were keyed by the random ARQ task uuid, so the lifecycle service's investigation-scoped pause/resume queries matched zero rows and fell through to weaker fallbacks. WorkflowStateCursor gains investigation_id + branch_id columns (migration 101, both nullable + indexed); the engine populates them at cursor creation from the run input, and pause/resume/purge query by them. resume drives its fan-out with the real branch id rather than the task uuid. The run_id ANY(...) legacy fallback is kept so investigations whose cursor predates this change still resume. Non-investigation workflows leave the columns null. Unit tests pin creation, the investigation-scoped find, resume restore with the correct branch id, and both legacy fallbacks. Live pause/resume on a running investigation is an operator smoke gate at merge -- this changes live lifecycle behavior and has no automated end-to-end coverage.
Extends the Unreleased entry with the increments completed in the faithfulness-remediation follow-through: the ResilienceLayer facade (RFC-07), the eval-gated self-improvement loop (RFC-08), the shadow/canary lifecycle stages (RFC-10), the generic McpClient with capability dispatch and pooling (RFC-11), the adaptive retrieval router with stable-core, graph, result gate, record-replay eval, and ingest enrichment (RFC-12), and the cursor-keying fix (RFC-02).
Faithfulness audit + full remediation complete -- dev at
|
Release 0.3.0 -- security, correctness, and reliability hardening
Rolls up the work landed on
devsince 0.2.1. Full detail inCHANGELOG.mdunder[0.3.0].Highlights
Behavior change requiring caller action
Not breaking
Verification
Opening for review; not for auto-merge.